Skip to content

feat(scripts): kernel-level axiom sweep with committed regression baseline - #300

Merged
dhsorens merged 9 commits into
mainfrom
alh/axiom-sweep
Sep 1, 2026
Merged

feat(scripts): kernel-level axiom sweep with committed regression baseline#300
dhsorens merged 9 commits into
mainfrom
alh/axiom-sweep

Conversation

@alexanderlhicks

@alexanderlhicks alexanderlhicks commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

What

Adds lake exe axiomsweep: kernel-level axiom / sorry accounting for the whole
library, backed by a committed regression baseline and an enforcing CI gate.

The tool loads the built .olean environment and computes the transitive axiom
dependencies of declarations under the swept root modules. This checks elaborated
kernel input rather than source text, so comments containing sorry do not create
false positives.

Why

CompPoly is a correctness-critical dependency of downstream provers. This PR makes
the library's current kernel-clean status mechanically checkable and prevents new
sorryAx or non-standard-axiom dependencies from silently entering CI.

How

  • scripts/AxiomSweep.lean: memoised traversal plus fixpoint repair for cycles;
    runtime-imports the built CompPoly environment and produces deterministic results.
  • scripts/axiom_baseline.json: records existing baselineable taint. New taint
    fails --check; removed taint stays green and prompts the baseline to shrink.
  • Native trust floor: bare Lean.ofReduceBool / Lean.trustCompiler and generated
    …._native.<tactic>.ax_* dependencies always fail and cannot be baselined.
  • CI: runs lake exe axiomsweep --check as an enforcing gate after the build.
  • Docs: documents the local workflow, CI behavior, generated baseline, and known
    blind spots.

Modes

lake exe axiomsweep                     # summary
lake exe axiomsweep --out report.json   # full per-declaration report
lake exe axiomsweep --check             # enforcing regression gate
lake exe axiomsweep --update-baseline   # refresh baselineable taint
lake exe axiomsweep --root Foo          # override roots (repeatable)

Current result

  • 7,588 declarations across 275 modules.
  • 0 sorryAx-tainted declarations.
  • 0 non-standard-axiom-tainted declarations.

Adversarial verification

  • A temporary sorry is detected and makes --check exit 1.
  • A generated native-style axiom is detected and makes --check exit 1.
  • --update-baseline refuses native trust, exits 1, and leaves the baseline unchanged.
  • The clean tree passes lake build, lake test, the axiom sweep, import checking,
    style lint, and documentation integrity checks on the current main / Lean 4.33.1.

Known limits remain documented: examples and structure-field defaults do not produce
ordinary reportable declarations, and unimported modules are invisible to an
environment walk; the repository's import-completeness check covers the latter.

🤖 Generated with Claude Code

alexanderlhicks and others added 3 commits August 10, 2026 12:35
…eline

Add `lake exe axiomsweep`: walks the compiled environment and computes,
for every CompPoly.* declaration, its transitive axiom dependencies —
the #print axioms information, library-wide, in one pass. Reads
elaborated .olean data, so private and macro-generated declarations are
included and no source heuristics are involved.

Baseline at this commit: 7589 declarations across 275 modules, 0
sorryAx-tainted, 0 non-standard-axiom-tainted — the library is fully
kernel-clean, and --check now keeps it that way (fails iff a declaration
is tainted that scripts/axiom_baseline.json does not list; the 6 'sorry'
tokens greps report on main are all inside comments).

Wire-up: report-only CI step in lean_action_ci.yml after the warm
rebuild; docs/wiki/quickstart.md documents the workflow.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…, docs

Collector: two-phase DFS + fixpoint repair. The one-pass DFS finalized
self-referencing constants (every inductive/ctor pair) prematurely and
memoized the wrong result for all later roots — confirmed by review to
produce rows diverging from #print axioms on sibling repos. The repair
pass re-derives every set in finalization order until stable: the least
fixpoint = true kernel closure, strictly more accurate than
#print axioms inside mutual families. Also: axiom *types* are traversed
(CollectAxioms parity), duplicate constNames rows deduped (7589→7573),
native trust-axiom names normalized to their owner (ax_N_M counters are
Elab.async/toolchain-volatile), --check/--update-baseline mutually
exclusive, unknown --root fails gracefully, nonstandard shrinkage
detected, and bare Lean.ofReduceBool/Lean.trustCompiler are never
baselinable (floor).

Docs: known blind spots documented (structure-field defaults and
examples never enter any environment walk; unimported files — paired
with check_imports); scope stated (tests/ and bench/ outside the
sweep); corrected sorry-token count (5, all comments); inventory rows
added to scripts/README.md, docs/wiki/generated-files.md, AGENTS.md
fast-start, quickstart CI mapping + lower-level commands.

CI: infrastructure failures (exit != 1) now fail the step; only taint
findings are report-only during the soak.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…teral for 4.30/4.31 portability

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

🤖 PR Summary

sorry delta: +4 (4 added) — proof obligations increased

feat(scripts): kernel-level axiom sweep with committed regression baseline


Statistics

Metric Count
📝 Files Changed 8
Lines Added 413
Lines Removed 2

Lean Declarations

✏️ Added: 18 declaration(s)

scripts/AxiomSweep.lean (18)

  • def buildEntries (roots : Array Name) : CoreM (Array Entry × Nat)
  • def currentBaseline (entries : Array Entry) : Baseline where
  • def dedupSort (a : Array String) : Array String
  • def defaultRoots : Array Name
  • def isReportable (n : Name) : Bool
  • def isStandard (a : String) : Bool
  • def kindOf : ConstantInfo → String
  • def neverAllowlistable (a : String) : Bool
  • def nonstandardOf (e : Entry) : Array String
  • def normalizeAxiomName (s : String) : String
  • def parseArgs : List String → Config → Except String Config
  • def reportNeverAllowlistable (cur : Baseline) : IO Bool
  • def runCheck (cur : Baseline) (basePath : String) : IO UInt32
  • def sorryAxName : String
  • def standardAxioms : List Name
  • partial def collect (env : Environment) (stack : List Name) (gray : Std.HashSet Name)
  • partial def repair (env : Environment) (order : Array Name)
  • unsafe def main (args : List String) : IO UInt32

sorry Tracking

Added: 4 `sorry`(s)

scripts/AxiomSweep.lean (4)

  • def currentBaseline (entries : Array Entry) : Baseline where (L229)
  • def runCheck (cur : Baseline) (basePath : String) : IO UInt32 (L258)
  • def runCheck (cur : Baseline) (basePath : String) : IO UInt32 (L259)
  • partial def repair (env : Environment) (order : Array Name) (L146)

📋 **Additional Analysis**

No findings.


📄 **Per-File Summaries**
  • .github/workflows/lean_action_ci.yml: A new 'Axiom sweep' CI step runs lake exe axiomsweep --check to verify that no unwanted axioms are introduced in the project. This step is placed between the timing report generation and the fail-on-warnings step, adding an additional validation layer to the build pipeline.
  • AGENTS.md: Added a new workflow step (step 6) to AGENTS.md instructing contributors to run lake exe axiomsweep --check when filling or adding a sorry, and to update the axiom baseline with --update-baseline if the change is intentional. This documents the project's practice of tracking axiom usage and ensuring that sorry additions are properly accounted for in the axiom baseline, with a note that native-compiler trust is never baselineable.
  • docs/wiki/generated-files.md: Added a new row to the table of generated files in docs/wiki/generated-files.md documenting scripts/axiom_baseline.json. This file is a generated and committed kernel-level axiom/sorry regression baseline, regenerated via lake exe axiomsweep --update-baseline after a lake build. It is checked by an enforcing axiom-sweep CI step, and the description notes that native-compiler trust cannot be baselined.
  • docs/wiki/quickstart.md: Adds a dedicated sub-section explaining how to use lake exe axiomsweep --check for kernel-level axiom/sorry accounting across the CompPoly.* library, including baseline management via --update-baseline and CI enforcement in lean_action_ci.yml (line added to the CI description). Also appends lake exe axiomsweep --check to the list of direct debugging commands near the end of the file.
  • lakefile.lean: Added a new lean_exe target named axiomsweep with srcDir := "scripts", root := AxiomSweep, and supportInterpreter := true. This executable performs kernel-level axiom/sorry accounting against a committed regression baseline (scripts/axiom_baseline.json). It runtime-imports the built CompPoly oleans, so it must be run after lake build`.
  • scripts/AxiomSweep.lean: Added scripts/AxiomSweep.lean, a new script that performs whole-library kernel-level axiom and sorry accounting for CompPoly.* modules. It computes the transitive axiom dependency set for every swept declaration via a DFS (collect) and a fixpoint propagation (repair), supporting four modes: bare summary, writing a full JSON report (--out), checking against a committed baseline (--check), and updating that baseline (--update-baseline). Key definitions include Entry, NonstandardEntry, and Baseline structures; helper functions normalizeAxiomName and dedupSort for robust name handling; reportNeverAllowlistable to reject native-compiler trust axioms; and runCheck which compares current taint sets to the baseline and reports regressions or resolutions.
  • scripts/README.md: The README now documents the new AxiomSweep.lean tool (invoked via lake exe axiomsweep), which performs kernel-level axiom/sorry regression checking against a committed baseline (axiom_baseline.json). A brief usage line lake exe axiomsweep --check was added to the script list, and the Script Inventory section gained a detailed sub-section describing the tool's modes (--check, --update-baseline, --out), its source from the built .olean environment, and its rejection of bare and native-compiler trust.
  • 1 file(s) filtered as noise (lockfiles, generated, or trivial): scripts/axiom_baseline.json

Last updated: 2026-09-01 19:09 UTC.

@github-actions

github-actions Bot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Build Timing Report

  • Commit: 79367c0
  • Message: Merge 07ce1c2 into ba5032e
  • Ref: alh/axiom-sweep
  • Comparison baseline: 8ad1aa2 from the previous successful PR update.
  • Measured on ubuntu-latest with /usr/bin/time -p.
  • Commands: library build (warm) lake build; test path lake test.
  • Clean build was skipped (warm/incremental CI; no toolchain or lake-manifest change). Runs automatically when lean-toolchain or lake-manifest.json changes, or via Actions → Lean Action CI → Run workflow with clean_build.
Measurement Baseline (s) Current (s) Delta (s) Status
Library build (warm) 1.51 10.66 +9.15 ok
Test path 1.00 4.35 +3.35 ok

Incremental Rebuild Signal

  • Warm-only run: default CI reuses cached oleans and rebuilds dirty modules only. No same-job clean:warm ratio.

Slowest Current Build Files (warm library build)

Showing 5 slowest of 5 repo targets parsed from the current warm library build log.

Wall (s) Path
3.80 CompPoly/Fields/Mersenne31/Fast.lean
1.70 CompPoly/Fields/Mersenne31/Basic.lean
1.30 CompPoly.lean
1.10 CompPoly/Fields/Mersenne31.lean
1.10 CompPoly/Fields/Mersenne.lean

@dhsorens

dhsorens commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

so, I added a sorry to test the axiom sweep in commit 530605d above. the check fails but the CI is still green. is this expected behavior @alexanderlhicks ? I would probably prefer it generate a comment or sth so that it reports something when it fails

@dhsorens

dhsorens commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

(removed the sorry in 6818d79)

@dhsorens

dhsorens commented Sep 1, 2026

Copy link
Copy Markdown
Collaborator

Fixed in 8ad1aa2: the axiom sweep is now an enforcing CI gate, so exit 1 fails the build. I also closed the native-trust baseline hole: generated ._native. axioms now always fail, and --update-baseline refuses to write them. Adversarial probes confirmed both failures and that the baseline remains unchanged.

@dhsorens dhsorens left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed on current main / Lean 4.33.1. The enforcing CI behavior, native-trust floor, clean build/test results, and documentation are all consistent. Ready to merge once required checks pass.

@dhsorens dhsorens left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-reviewed after merge commit 07ce1c2. The axiom-sweep changes are unchanged; lake build, lake test, and the enforcing sweep all pass on the updated head (7,699 declarations, zero taint).

@dhsorens
dhsorens enabled auto-merge (squash) September 1, 2026 19:08
@dhsorens
dhsorens merged commit bfea0dc into main Sep 1, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants